Search AI Products and News

The rapid advancement of generative AI technology has revolutionized digital content creation, but bias remains a persistent issue. A recent investigation by Wired magazine into OpenAI's video generation tool, Sora, revealed that the tool frequently reinforces stereotypes related to race, gender, and ability, sometimes even neglecting the representation of certain groups. Image Note: Image generated by AI, image licensed by Midjourney. In the investigation, researchers provided S...

OpenAI researchers recently discovered an interesting phenomenon: the usernames chosen by users when interacting with ChatGPT may subtly influence the AI's responses. However, this influence is generally very small and primarily limited to earlier, unoptimized versions of the model. This study delves into the differences in ChatGPT's reactions to identical questions when faced with usernames related to different cultural backgrounds, genders, and ethnicities. The choice of usernames as a focal point for the research was made because names often carry specific cultural, gender, and ethnic connotations.

Recently, OpenAI's research team discovered that the usernames chosen by users can somewhat influence the responses of ChatGPT during interactions. Although this effect is relatively minor and primarily seen in older models, the findings are still noteworthy. Users typically provide their names to ChatGPT for tasks, making the cultural, gender, and racial backgrounds inherent in those names an important factor in studying bias. In this research, the researchers explored ChatGPT.

Recently, a new study has drawn attention, revealing that ChatGPT's responses reflect the cultural values of people from English-speaking and Protestant countries. The core of this study is that large language models like ChatGPT were trained on a vast amount of data from certain countries and cultures, which may lead to outputs that carry cultural biases. The research team, led by René F Kizilcec and his colleagues, examined five different versions of OpenAI's GPT.

Language models have shown widespread applications across various fields, from education to legal consulting, and even in predicting medical risks. However, as these models gain more weight in decision-making processes, they may unintentionally reflect the biases present in the human training data, exacerbating discrimination against minority groups. Research has found that language models exhibit implicit racism, particularly in their treatment of African American English (AAE), demonstrating harmful dialect discrimination that is more negative than any known stereotypes against African Americans. The 'masking disguise' method was used to compare AAE with Standard American English.

The Oklahoma City Police Department is experimenting with the AI chatbot 'Draft One' to generate reports in order to save time. Developed by Axon, the system utilizes ChatGPT technology to quickly produce reports from audio recordings. Supporters believe this could significantly improve efficiency, but critics are concerned about potential errors, biases, and the risks of power abuse that automation could bring. Activist Aurelius Francisco expresses discomfort with the use of AI and tasers from the same company, fearing potential misuse.

The application of artificial intelligence in job recruitment aims to improve objectivity and efficiency, but studies reveal its potential to exacerbate bias. AI learns from data, and biases in datasets can lead to unfair algorithmic outcomes. The educational and professional gaps between HR and AI developers hinder effective collaboration, impacting the ability to identify and reduce biases. To address this issue, it is recommended to implement structured training programs to enhance HR professionals' knowledge of AI; promote interdisciplinary teamwork to ensure datasets are diverse and represent different groups; and establish guidelines and ethics.

Black founders have noticed that mainstream large language models like ChatGPT have significant shortcomings in addressing cultural differences. These models often provide overly generic responses to questions unrelated to a Western cultural context, lacking an understanding of African Americans and their unique cultural traditions.

Experiments have found that OpenAI GPT-3.5 exhibits racial bias in resume screening. Test results indicate that GPT-3.5 is more inclined towards certain ethnic names, which could impact hiring decisions. The experiments also revealed gender and racial preferences for GPT in different job positions. OpenAI responded that companies usually take measures to mitigate bias when using its technology.

Google CEO strongly condemns the bias issues in Gemini AI applications, particularly the controversies surrounding the image generation of historical figures. Sundar Pichai has promised employees to fully address the Gemini issues, leading Google to suspend the use of the Gemini image generator.

Users criticized Gemini for generating diverse images in inappropriate historical contexts. Google urgently fixes the AI tool to avoid accusations of racial discrimination, as the Gemini bot had previously faced diversity issues.

Experts indicate that OpenAI's latest video generation tool, Sora, may contribute to misinformation and bias. Sora has showcased fake videos of Dalmatian dogs, Chinese New Year parades, and archaeologists digging chairs. Experts are concerned that Sora will exacerbate the issue of false information and government propaganda on the internet, blurring the lines between real and fake content. OpenAI has taken measures to prevent the misuse of Sora, but experts remain worried about its potential for deepfake video capabilities.

The U.S. Department of Defense has recently launched a bounty program aimed at identifying legal biases in artificial intelligence models. The program requires participants to extract clear evidence of bias from Meta's open-source LLama-270B model. Through this initiative, the Pentagon hopes to...

Generative artificial intelligence is expected to create nearly $2 trillion in wealth for the U.S., with $50 billion flowing to families. Black families 'earn 38 cents for every additional dollar of household wealth,' and if accurate, the racial wealth gap could widen by $43 billion. Black professionals mainly work in jobs that do not require degrees, such as office support, manufacturing, food service, and mechanical installation and repair, which may be replaced by AI, further exacerbating the wealth gap. Generative AI is projected to create nearly $2 trillion in wealth in the U.S.

Researchers evaluated the racial and gender bias of GPT-4 in clinical decision-making and found significant biases. The study calls for bias assessments on large language models. The findings were published in The Lancet Digital Health.